1. Field of the Invention
The present invention relates to an image processing apparatus, an image processing method, and a program. More particularly, the present invention relates to an image processing apparatus and an image processing method capable of performing interpolation for a mosaic image with higher accuracy, and a program therefor.
2. Description of the Related Art
When an image is captured using an imaging apparatus incorporating a single-plate solid-state imaging device, only single spectral sensitivity can be obtained. For this reason, in general, a method of capturing color images, in which a different color filter is placed on each of a large number of pixels, and those color filters are arranged in a specific pattern, has been used. In an image captured by this method, an image in a mosaic form with respect to colors is generated. Therefore, by performing interpolation by using color information obtained from adjacent pixels, an image in which all the colors (for example, RGB) are available in all the pixels can be generated.
In general, for the arrangement of a filter, a Bayer arrangement shown in FIG. 1 is used. The Bayer arrangement is formed in such a manner that a total of four filters, that is, two G filters that allow light of only green color (G) to pass therethrough, one R filter that allows light of only red color (R) to pass therethrough, and one B filter that allows light of only blue color (B) to pass therethrough, define a minimum unit. That is, the G filters are arranged in a checkered pattern and exist at a density twice that of the R and B filters. Therefore, a method is often used in which, for G having a large amount of information and a strong correlation with luminance, G are made available for all the pixels, and a pixel interpolation process is performed by using G as reference information, thereby allowing information on the other colors to be available in each pixel.
For example, in U.S. Pat. No. 4,642,678 (Cok, David R.), a pixel interpolation method has been proposed in which, under the assumption that the ratio of colors in a local area is almost constant, G is made available in advance in all the pixels, the average of the ratios of R to G and the average of the ratios of B to G in adjacent pixels are multiplied by G at the position of the pixel of interest, in order to estimate color components which are not known. In such a pixel interpolation method in which G is used as reference information, there is a problem in that the final result greatly depends on the interpolation accuracy when G is made available for all the pixels.
Therefore, in U.S. Pat. No. 5,382,976 (Hibbard, Robert H.), a scheme in which, based on the gradient value of G, interpolation accuracy is improved by switching adjacent pixels of G used for interpolation, has been made. Furthermore, in Japanese Patent No. 2931520, a method has been proposed in which, in addition to switching adjacent pixels of G used for interpolation, horizontal and vertical interpolation values at the position of the interpolation pixel are obtained, and two interpolation values obtained by a process suitable for a case in which a correlation in the horizontal direction is strong and a process suitable for a case in which a correlation in the vertical direction is strong are mixed using correlation values.
The above-described various methods are assumed on the features of the Bayer arrangement such that G is arranged in a checkered pattern and the sampling frequency is higher than that for the other colors. However, the filter of the single-plate solid-state imaging device is not limited to three colors, and four colors may be used. In other than the filter arrangement of the so-called complementary color system, for example, in Japanese Unexamined Patent Application Publication No. 2002-271804, the formation of a filter by the arrangement of four colors by further adding another color (for example, Y) to the three colors of RGB for the purpose of improvement of color reproduction has been proposed. Also, for example, in the Bayer arrangement, there are cases in which two Gs within a 2×2 filter arrangement do not always have the same characteristics due to, for example, differences in the arrangement of R and B adjacent to G, differences in the spectral characteristics caused by position deviations of filters in a manufacturing process, disclosed in Japanese Unexamined Patent Application Publication Nos. 7-135302 and 8-19114, and an error of gain between lines, which occurs in a solid-state imaging device having a different signal output between even-numbered lines and odd-numbered lines. Actually, an arrangement of four colors is formed.
However, the method of interpolating G in all the pixels, in which a checkered arrangement of G is used in the Bayer arrangement, cannot be applied to a signal obtained from such a filter arrangement of four colors. Naturally, there is a problem in that the color interpolation method on the assumption that G is made available for all the pixels cannot be used as is.
Furthermore, even if G or signals replaced with G are made available by some method, in the method, proposed in Japanese Patent No. 2931520, in which the correlation values in the horizontal and vertical directions calculated at the position of the pixel of interest are used as is, there is a problem in that, if correlation determination is incorrect due to influence of noise, etc., in the vicinity of a limit resolution, the two interpolation values are mixed irregularly, and a very noticeable artifact occurs.